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Program overview 


Online School Partnerships 


The MacPhail Center for Music Online School Partnerships program aims to increase access 
to high-quality music education for students who attend schools in Greater Minnesota in 
order to improve student interest and knowledge in music, and to improve or maintain 
positive school engagement and non-academic skills. The MacPhail Teaching Artists work 
collaboratively with school music educators to develop curriculum, implement the Online 
School Partnerships program, and work with students individually, in sectionals, or in large 
group settings on their music lessons via the internet. Teaching Artists teach classes at the 
partner schools remotely and in real time through teleconferencing technology. Classes are 
typically 30-60 minutes in length, with a MacPhail Teaching Artist working with students 
on their repertoire for their ensemble or for an audition, as well as helping students as they 
develop their musical skills. All of the session content is based on what the partner schools’ 
music educator would like the Teaching Artist to cover — the Teaching Artist then creates 
the session structure. 


The Online School Partnerships program has been providing lessons and working with 
schools outside the Twin Cities metro region since 2011. In 2020-21, it had 23 Teaching 
Artists, partnered with 14 schools, and served more than 988 students. Schools involved 
in the partnership had a wide range of instructional time, between 4 and 291 hours for each 
school, totaling 1,352 hours of instruction; schools may choose the type of instruction 
(individual, sectional, or classroom) that makes sense for them, given the number of students 
they would like to have participate. This evaluation, however, focuses on ten schools who 
chose to participate in the evaluation in the 2020-21 school year by completing music skills 
assessments and student surveys: Buffalo High School, Detroit Lakes High School, John 
Marshall High School, Kingsland High School, Kerkhoven-Murdock-Sunburg High School, 
MACCRAY High School, Mayo High School, Pierz Healy High School, Rush City High 
School, Tri-County High School, Willmar High School, and YME Middle and High School. 
This evaluation focuses on individual and small group lessons with a MacPhail Teaching 
Artist. The types of lessons provided by the Online School Partnerships program are: 


= Group lessons: These lessons are ongoing and built on method books, handouts, and 
sequential practice planning. The MacPhail Teaching Artists consult with the partnering 
music educators to create ongoing assignments that encourage musicianship and solid 
playing techniques for the group of students, with a focus on one particular instrument 
or voice type. 
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= Sectionals: In these lessons, MacPhail Teaching Artists work to boost the skills of 
specific sections of choir, band, or jazz bands. They work on repertoire that participating 
students are playing in large group. These students work on development of sound, 
rhythmic precision, general musical understanding, and performance skills. 


= Individual lessons: These lessons offer one-on-one instruction with an instrument or 
vocal specialist to work on general tone and technique. Teaching Artists may also 
provide support as a student prepares for All-State band or choir and contest preparation. 


Sixty-eight students took lessons through the Online School Partnerships program at the 
schools involved in this evaluation and all took individual lessons from Teaching Artists; 
no students took group or sectional lessons. 


Sing Play Learn 


Sing Play Learn with MacPhail® Online Early Childhood Music Partnerships strives to 
increase access to high quality music education for early childhood students in Greater 
Minnesota. Student participants benefit developmentally with gains in executive functioning, 
social-emotional skills, foundation in musical concepts, and cognition and language 
development through musical learning experiences. The virtual musical play experiences 
are customized to support classroom goals via collaboration with the early childhood 
classroom teachers and taught by MacPhail-trained early childhood music specialists. 
Classes range from 20-30 minutes with live, interactive musical activities. Additional 
opportunities include teacher workshops on music integration and parent education on the 
benefits of music on learning and development. 


The program began serving Greater Minnesota in 2013. In 2020-21, the program had three 
Teaching Artists. It partnered with four schools and served 179 children. They provided a 
total of 139 hours of instruction overall. Partner schools included early childhood programs 
within the school districts of Gibbon-Fairfax-Winthrop, Kerkhoven-Murdock-Sunburg, 
New Richland-Hartland-Ellendale Geneva, and also Laker Prep, located in Detroit Lakes. 
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Evaluation approaches 


Online School Partnerships 


Wilder Research worked with the Online School Partnerships program to develop an 
evaluation design for the program. The evaluation includes implementation and outcome 
components, and gathers qualitative and quantitative data to answer the following research 
questions. (The Online School Partnerships program developed a logic model and these 
research questions reflect program activities and outcomes. See Appendix). 


Implementation evaluation key questions 


= How well is the Online School Partnerships program being implemented? What are 
aspects of the program, if any, that need to be adjusted over time? 


= How well does the collaboration between MacPhail and the schools function, and 
how can it be strengthened? 


Outcome evaluation key questions 


= To what extent does student participation in the Online School Partnerships program 
affect their interest, motivation, knowledge, and skills in music? 


m= To what extent does student participation in the Online School Partnerships program 
affect their attitudes about school (school motivation and engagement) and student 
non-academic skills? 


= To what extent does music educator participation in the Online School Partnerships 
program affect their growth as an educator, job satisfaction, and ability to handle many 
aspects of their work assignment? 


Methods 


MacPhail and Wilder Research staff worked collaboratively to complete this evaluation. Both 
MacPhail and Wilder staff completed data collection responsibilities, and Wilder Research 
staff completed data entry, data analysis, and reporting. 


The following are the data collection instruments used in the evaluation. 


m Teaching Artist survey: In spring 2021, Wilder conducted a web survey with the 
MacPhail Teaching Artists. This survey asked about the overall vision and goals of 
the program, the collaboration with the partner schools, successes of the Online School 
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Partnerships program, and areas for improvement. Twenty-three Teaching Artists 
completed the survey, for a response rate of 100%. 


Music educator interviews: Wilder conducted phone interviews with six school music 
educators involved with the Online School Partnerships program. In these interviews, 
educators were asked about the value and successes of the program, collaboration 
with MacPhail, and suggestions for improvement. Six of ten eligible educators were 
willing to be interviewed for this project (60%). 


Student survey: This survey was developed to measure student satisfaction with the 
Online School Partnerships program, perceptions of the lessons provided, and interest 
in music education and school generally. In spring 2021, 102 ninth- through twelfth- 
graders completed a survey, representing 24% of the 419 students in the program at 
schools that distributed the student survey. 


Music skills assessments: At the beginning and end of students’ time with the Online 
School Partnerships program, the MacPhail Teaching Artist they worked with completed 
skills assessments, looking at a variety of components of students’ performance abilities 
(tone quality, intonation, rhythm, etc.). Pre-post music skill assessments were completed 
for 41 out of 216 students (19%) involved in the Online School Partnerships program 
at schools that conducted the music skills assessment. 


Parent interviews: Wilder conducted phone interviews with two parents of children 
involved in the Online School Partnerships program. In these interviews, parents were 
asked about their family’s satisfaction with the program, what they believe their child 
has gotten out of the experience, and how they believe the program could improve, 
among other things. 


Sing Play Learn 


Wilder Research worked with the Sing Play Learn program to develop an evaluation 


design for the program. The evaluation includes implementation and outcome components, 


and gathers qualitative and quantitative data to answer the following research questions. 


Implementation evaluation key questions 


Do child development centers benefit from their partnership with the MacPhail Center 
for Music? 


Do classroom teachers and Teaching Artists develop useful and meaningful 
relationships? 
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Outcome evaluation key questions 
= To what extent does participation affect students’ listening skills? 
= To what extent does participation affect students’ self-regulatory skills? 


= Does students’ level of engagement with the lessons grow over the course of the year? 


Methods 


MacPhail and Wilder Research staff worked collaboratively to complete this evaluation. 
Wilder Research designed the data collection tools in partnership with MacPhail, and Wilder 
Research staff conducted data collection, analysis, and reporting. 


The following are the data collection instruments used in the evaluation. 


= Classroom Teacher Survey: In spring 2021, Wilder conducted a web survey of 
classroom teachers who partner with the Sing Play Learn program. This survey asked 
about the overall vision and goals of the program, the collaboration with the MacPhail 
Center for Music, successes of the Sing Play Learn program, and areas for improvement. 
Four classroom teachers completed the survey. 


m= Teaching Artist Interviews: Wilder conducted phone interviews with four Teaching 
Artists involved with the Sing Play Learn program. In these interviews, Teaching 
Artists were asked about the value and successes of the program, collaboration with 
the partner schools, and suggestions for improvement. All three Teaching Artists 
participated in an interview, for a response rate of 100%. 
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Results — Online School Partnerships 


This section describes the implementation and outcome results for the Online School 
Partnership program. Results are reported across the schools given the small number of 
responses from each school. 


Implementation 


Successes 


Students enjoy their lessons, particularly working with Teaching Artists. 


Students were asked about their satisfaction with the program in a survey administered at 
the end of the school year. Of the students who answered the survey, 73%-83%agreed or 
strongly agreed that they liked coming to the online sessions (Figure 3), that they would 
recommend the program to other students (Figure 3), and that the technology used for the 
sessions made it easy to learn (Figure 4). 


Similarly, students reported positive perceptions of and interactions with the Teaching 
Artist they worked with. Nearly all respondents said they liked their Teaching Artist and 
that their Teaching Artist used class time well, while 81% said the Teaching Artist set clear 
expectations for practicing and learning at home (Figure 5). 


Students were asked in an open-ended question what they liked most about participating 
in the Online School Partnerships program (Figure 7). Students most frequently mentioned 
improving their musical skills, such as learning new skills or techniques. They also 
mentioned having positive interactions with the Teaching Artist, stating that they were a 
good teacher or just liking them in general. 


This program helped me learn and improve, and I still had fun and was able to 
understand how to do new things. -OSP Student 


It helped me gain confidence in my abilities. -OSP Student 


The instructor was very engaged and offered technological tools to improve my 
musicianship. -OSP Student 


I liked how even though we were distanced you still grow a connection with your 
lesson teacher. -OSP Student 


In addition, some students reported a few issues with their Teaching Artist, stating that 
the Teaching Artist had some troubles using the online technology platform sessions, had 
poor internet connections, or was not fully present in the lesson as they delivered the lessons 
from home. 
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MacPhail is a resource and partner to schools and students in Greater Minnesota. 


Teaching Artists were asked to describe what they believe the value of participating in 
the Online School Partnerships is for partner schools. Nearly all spoke about the program's 
impact on participants, specifically how the program increases access to high quality 
musical instruction that is otherwise unavailable to students. In particular, they noted that 
students have access to instructors who specialize in their instrument, rather than only 
having access to music instruction from generalists. 

I love working with the students; sometimes they are from struggling band programs, 

other times they are from a strong program —everyone is eager to learn. I can bring 

individual help and my specialized knowledge of clarinet technique and performance 


to the students. Band directors are required to be generalists, so my skills enhance 
their leadership. —Teaching Artist 


Several Teaching Artists discussed the ways in which the program is a resource to music 
educators and schools. Music educators are able to learn new skills, methods, and approaches 
by observing lessons provided by the Teaching Artists. Teaching Artists are also able to 
support music educators in building more skilled student ensembles. Some noted that there 
are ripple effects of instruction, as individual students go back to their ensembles with new 
skills and techniques which increase the quality of the ensemble and allows the student to 
share with others what they have learned. 

This program is an invaluable resource to music educators throughout the state. 

Through our online school partnerships program we are able to provide students 

with an unprecedented level of access to highly trained specialists. This frees up 

music educators to focus deeper into music literacy, ensemble skills, and artistic 

expression rather than focusing on esoteric pedagogy on instruments they are 


less familiar with. Students are provided with a more in depth education from 
both their individual teachers and their classroom instructors. —Teaching Artist 


Other benefits of participating include students' increase in their musical confidence and 
allowing students an opportunity to interact with others outside of their own community, 
potentially broadening their view of the world and what is available to them. 


It gives the students a chance to interact with someone from outside their immediate 
community. This interaction may help broaden the students' worldview. 
—Teaching Artist 


In interviews, parents spoke about the importance of the program in giving their children 
access to music education that would not otherwise be available to them. They spoke about 
how their children were able to pursue music at the collegiate level, which otherwise might 
not have been possible if not for the advanced instruction they received through the Online 
School Partnerships program. 

This is just an amazing opportunity for out-of-state kids. And I think it helps them 

compete better, with the kids who are in the Cities. Because [students who live in 

the Cities] kind of have a leg up — they’ve got proximity to MacPhail and there 

are more teachers available. So maybe it helps level the playing field... I don’t 

know that [my child] would have had this awareness or skill level without MacPhail. 

I don’t think he would have competed at the All State-level or competed for college 

scholarships without MacPhail. —Parent 
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Music educators were asked to describe the benefits of participation in the Online School 
Partnerships for their school as a whole and how MacPhail serves as a resource to their 
school. Educators most commonly spoke about how the Teaching Artists serve as great 
resources to them when they have questions. For example, some music educators reached 
out to Teaching Artists with questions about curriculum, and some asked questions when 
students were looking to purchase a new instrument. Music educators also spoke about 
how MacPhail’s coordination of the program was valuable to them, as they are stretched 
too thin with their existing work to do the type of coordination it would take to connect 
students to these types of lessons. Others spoke about how communication is strong between 
them and the Teaching Artists they work with and how Teaching Artists have provided 
other types of materials that are beneficial to students’ learning. 


Music educators also spoke about how this program benefits them; Teaching Artists serve 
as a resource for music educators, who can feel relatively isolated from their peers given 
their geographic distance to other music programs. Music educators can get feedback 
about their instruction from Teaching Artists, or turn to Teaching Artists with questions 
about instruments and curriculum. Music educators also noted that they benefit from the 
help in teaching the large number of students they see; while the music educator is not 
able to step away from the large group to focus on a particular skill or section of music, 
they are able to connect those students to that opportunity through the Online School 
Partnerships program. Many music educators use the program as an opportunity to connect 
the strongest players with additional opportunities so that they have the opportunity to grow 
through private, intensive lessons that may not otherwise be available to the student because 
of cost or distance to instructors. 


Sometimes he’ll catch me not interpreting the music correctly. It’s something 
where I was interpreting the beats slightly wrong. He was right, in retrospect. 
What I was doing was working, but it was better his way—it made the performance 
more accurate. He’s helped me as well, because as a teacher, I sing along. It’s 
training for me as well, which I appreciate. It’s helped me. —Music educator 


I learned things as I sat in. It gave me more tools in my toolbox. Like, “hey, let’s 
try this.” I was able to incorporate it into my classroom. —Music educator 


Music educators and Teaching Artists build relationships with one another. 


In interviews, music educators were asked if they were able to build relationships with 
the Teaching Artists who instruct their students. All educators were very positive about 
the relationships they had built with the Teaching Artists they work with. They discussed 
the ways Teaching Artists were supportive and went above and beyond to help music 
educators and students, including listening to recordings of students playing outside of 
scheduled sessions, answering music educators’ questions, and communicating with music 
educators about their goals and priorities for students. Some music educators commented 
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that they were able to have more communication with Teaching Artists when school was 
fully remote, as schedules got more hectic as students returned to school. 
Before we started the program, I did a lot of communicating with [our Teaching 


Artist], saying, ‘here are the goals, the parameters, and our class schedule.’ We 
had a lot of communication.” —Music educator 


Teaching Artists also spoke about how they have forged strong relationships with partner 
schools, with one commenting that their work together has gradually changed the culture 
of the partner school's music program for the better. 

Our collaborations with schools have grown more successful by the year. The 


longer we work with individual programing the more the culture of a school's 
music program changes. A rising tide raises all boats. -Teaching Artist 


MacPhail supported students and educators during the shift to distance learning. 


Public schools in Minnesota continued to use distance learning throughout most of the 
2020-21 school year because of the COVID-19 pandemic. Music educators were asked 
how MacPhail supported their efforts in this second school year affected by the pandemic. 


Music educators were positive about the support MacPhail provided as they continued to 
utilize distance learning, and for many, come back in-person at the end of the school year. 
Some noted that it was helpful that they had already worked with MacPhail prior to the shift 
to distance learning; because relationships and processes were already in place, it was easier 
to make the transition to students holding their lessons at home, or for music educators to 
make a plan for students to come into the school for their lesson. While the Online School 
Partnerships program inherently operates remotely, as Teaching Artists are based out of the 
Twin Cities metro and participating students live in rural areas of Greater Minnesota, it can 
still be challenging for students to participate in lessons from home. Many areas of Greater 
Minnesota do not have high quality internet access, especially in residential settings. These 
students benefited from having the option to come into the school building for their lessons. 


Some music educators commented that the Online School Partnerships program, as well as 
their own school-based programs, are typically focused on preparation for concerts. Because 
concerts were cancelled during the pandemic, music educators and Teaching Artists had to be 
flexible in their teaching approach, since they had previously worked through concert pieces 
during their time together. 

At the time we started our MacPhail partnership, we were in distance learning 

without any concerts on the horizon at all. I think maybe that threw some of the 


MacPhail staff off a little bit, because it’s just something they hadn’t been used 
to. —-Music educator 
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When asked, all Teaching Artists agreed that they had enough information and guidance 
from MacPhail program leaders to support them during distance learning in the 2020-21 
school year. Teaching Artists were asked how MacPhail program leaders can better support 
them as they teach students who are doing distance learning. Most commonly, Teaching 
Artists commented that they felt supported and had no suggestions for improvements. 
Teaching Artists also spoke about how important communication is; several noted that 
they would simply like MacPhail to continue with the level of communication they have 
already established, and a couple of Teaching Artists suggested that there be more support 
in connecting Teaching Artists to the schools they serve, either by having an email or cell 
phone number for someone at the school they can contact on short notice, or having it 
established early on in the partnership that the Teaching Artist might directly reach out to 
the school if necessary. 

T really appreciate how lessons continued for throughout the pandemic, even from 


students’ homes. Continuity was maintained which I was important for them and me! 
—Teaching Artist 


I think having a clear line of open communication would be helpful. While I did 
feel confident reaching out, I didn’t feel like the line was open directly to me. 
—Teaching Artist 


After this, Teaching Artists suggested MacPhail continue to search for technologies that 
allow for smooth facilitation and scheduling of online lessons. Other suggestions include 
MacPhail providing more support for Teaching Artists who shift the types of lessons they 
provide (from individual to ensemble lessons, for example), that MacPhail continue 
online-only performances after the pandemic is over, and that MacPhail provide a toolkit 
of best practices for online lessons to be shared with partner schools. 


Students who indicated they had completed at least some or all of their online lessons at 
home over the year were asked about their experience (Figure 6). Nearly all agreed their 
internet at home was high enough quality and reliable enough for online lessons and that 
they had available space to participate in the lessons at home. Approximately half of students 
agreed having lessons at home was as easy as having lessons at schools but over 38% 
disagreed with the statement. 


Opportunities for improvement 


Teaching Artists and students wish to have better internet connections, equipment, 
and technology support. 


When asked how the program could improve, a few Teaching Artists and students mentioned 
difficulties with technology and equipment (see Figure 8 for student responses). Both groups 
noted the need for improved internet connection in particular, as well as technical issues 
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with Skype and the need for improved microphones. While only a few mentioned these 
technology issues, they seem essential to delivering online instruction. 


I think continued research into new technologies will be key to continued 
improvement. —Teaching Artist 


Opportunities for occasional in-person visits would strengthen relationships. 


When asked how MacPhail could improve its collaboration with partner schools, Teaching 
Artists most frequently suggested there be opportunities to meet in person with the students 
they teach. They suggested that this could happen either by visiting the communities in 
Greater Minnesota that they work with, or by having funding available for students to come 
to the Twin Cities to meet and practice with their Teaching Artist. Some music educators 
also spoke in interviews about their desire to have in-person visits with the Teaching Artists 
they work with. 

I would love to have the resources to visit the schools in person, at least once a 

year for each partnership. Sometimes schedules don't connect, but when I have 

been able to visit the students, our connection has been improved. The students 

feel valued because we have taken time to travel to meet them. We learn about 

their school and their community, which makes us better teachers. Even before 


the pandemic, I found value in confirming that what I hear online is what I hear 
in-person. When we visit, the students can learn the same. —Teaching Artist 


In interviews, one parent spoke about how it would be nice for participants and their 
families to have the opportunity to meet their Teaching Artist in person to further solidify 
the connection between the two. 


Expanding curriculum with different styles and genres would enrich the program. 


A couple of Teaching Artists suggested changes to the curriculum. Specifically, they noted 
that it would be beneficial to include music theory and relevant history and literature lessons 
alongside their traditional lessons, or that it would be beneficial to offer different styles 
and genres of music, noting that different instructors who have that knowledge would 
likely need to be brought in to the program by MacPhail. 

It would be great to offer sessions on different styles and genres of music through this 


program. Funding to bring in teachers versed in diverse genres would be needed. 
—Teaching Artist 


Improved communication and relationship building would improve collaboration. 


Teaching Artists also spoke about ways to improve the relationship between themselves 
and music educators in the program. A couple of Teaching Artists suggested ways that 

communication between them and the partner schools could be improved: one suggested 
that Teaching Artists and music educators have regularly scheduled meetings, and another 
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suggested that music educators copy Teaching Artists on emails to their band, orchestra, 
and choir students so that the Teaching Artist is kept in the loop. One noted that they would 
like their partner schools to view them as a long-term support in creating well-rounded 
student musicians, rather than a one-off coach. 

If the partnership ensemble teachers included the MacPhail instructors on any 

group emails that they sent out to their band or orchestra or choir students, that 

could help MacPhail faculty stay informed as to what each students' ensemble 

director is prioritizing that week and in the near future to be able to better focus 


lesson time on repertoire or skills that the ensemble is focused on as a whole. 
—Teaching Artist 


Additional resources would benefit music educators and students and their families. 


Music educators were asked what other tools or resources MacPhail could provide to help 
music educators and schools in Greater Minnesota. Overall, music educators were very 
positive about the tools and resources already available to them through the program; some 
noted that they should do more to utilize everything already available to them through 
MacPhail. Some educators noted that funding the program can be challenging for them. 
Others noted that video tutorials for students are beneficial and requested that MacPhail 
create these. For example, one music educator requested that Teaching Artists create a 
tutorial for students about how to clean their instrument. 

One thing is if the instructors ever came up with how to clean your instrument 


videos... That would be an incredible time saver. I have high schoolers 26 minutes 
each day. We don’t have much time. We can’t clean during class. — Music educator 


Parents were also asked what resources MacPhail could provide that would be beneficial 
for their families. One parent suggested that Teaching Artists offer more guidance on the 
process students would go through to apply for music scholarships as they go through the 
college application process, including how to create a music resume. One parent suggested 
that MacPhail offer financial support to families to help them pay for instruments, which 
can be difficult to afford for many families. 


Setting appropriate expectations can be challenging for Teaching Artists. 


Teaching Artists noted that it can be hard to set reasonable expectations for students who 
are not as motivated to improve their skills as others; one Teaching Artists noted that they 
want to push the student to progress, but not too hard so as to push the student to quit. 
Some students move pretty slowly in their progress. I sense if you clamp down 
and insist on more progress, they likely would quit, so maintaining an interest in 


the lessons is the highest priority. But it seems for some, itis a lower expectation 
at the high school level. -Teaching Artist 
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Other programmatic recommendations varied. 


Teaching Artists provided a wide range of ideas to improve the impact of the program. 
These included enrolling more students, holding more lessons, reducing the paperwork 
burden for Teaching Artists, and strengthening funding for the program for the full 
academic year. 


Some music educators mentioned that the program has grown and evolved over time, and 
that MacPhail has been open to learning from experience and adapting as new resources, 
especially technology, becomes available. 


Some music educators noted that Teaching Artists sometimes have higher expectations 
than may be reasonable for high school students, and that they should work to find a happy 
medium in which they are still pushing students to learn and grow but not putting too much 
pressure on them. 


Many music educators did not offer suggestions to improve the program, instead praising 
MacPhail for how flexible and accommodating they are with the program. Many spoke 
about how helpful it is to have MacPhail coordinate logistics for the program and mentioned 
specific staff and Teaching Artists who have made participation in the program easy, useful, 
and pleasant for their schools. 


Outcomes 


Benefits to students in music 


Students gain musical skills and knowledge. 


When students were asked to how much they agree with the statement “I learned new 
musical skills that helped me in band/choir,” students were overwhelmingly positive; 
28% of respondents strongly agreed with the statement and another 56% agreed (Figure 9). 


Teaching Artists assessed students’ musical skills at the beginning and end of their time 
with the Online School Partnerships program. Students were assessed on a scale of 1-5 ina 
variety of areas: tone quality, intonation, rhythm, technique, musical expression, bowing (for 
string students) execution, articulation, and music reading skills (Figures 10-18). 


On average, students were rated highest in rhythm (3.6), followed by tone quality, intonation, 
and execution (3.5 in all; Figure 1) at post assessments. Staff ratings show that on average 
students moved from ‘good’ at the beginning of the program to between ‘good’ and 
‘excellent’ at the end of the program. 
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1. Mean musical skill scores, pre-program to post-program 


Pre Post 
Rhythm 3.0 @mmmm® 3.6 
Tone quality 3.0 @mmm@® 3.5 
Articulation 2.9 @mmm® 3.4 
Execution 28@ @ 3.5 
Intonation 28©@ @ 3.5 
Bowing (strings) 28©@ @3.4 
Reading skills 2.8 Omm® 3.2 
Musical expression 27@ @ 3.4 
Technique 2.7@ @3.4 
1 2 3 4 5 


Note. This chart shows mean scores at pre- and post-assessments of all Online School Partnerships program 
participants. 1=poor, 2=fair, 3=good, 4=excellent, and 5=superior. 


Teaching Artists assessed students’ musical confidence and ability to focus before and after 
the program (Figures 19 and 20). On average, students made improvements in confidence 
(from mean scores of 3.0 to 3.7) and in ability to focus (from mean scores of 3.7 to 4.0; 
Figure 2) from pre- to post-assessments. 


2. Musical attitude 


Pre Post 
Ability to focus 3.7 @m@® 4.0 
Confidence 3.0@ @ 3.7 
1 2 3 4 5 


Note. This chart shows mean scores at pre- and post-assessments of all Online School Partnerships program 
participants. 1=poor, 2=fair, 3=good, 4=excellent, and 5=superior. 


Teaching Artists also had the opportunity to include additional comments to the post- 
program assessments. Nearly all comments were positive, noting musical changes or 
improvements in students, that students were a pleasure to work with, or students were 
able to audition for competitions or college scholarships. 


The All-State audition materials were quite a step up in difficulty. [This student] 
learned a great deal, and while she didn't play them perfectly, her improvement 
during the year was significant. 


[This student] is a wonderful success story. Over the past two years that we've 
worked together, he has transformed from someone who passively plays in band 
to some who actively enjoys music and strives to improve. This has been excellent 
for his self-confidence and his overall sense of self. 
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The student prepared for college entrance auditions, and the schools she applied 
to included scholarship eligibility in the entrance audition process. She won a music 
scholarship to [local college]. 


Parents were asked in interviews if they had seen changes in their child’s musical skills 
because of their participation in the program. Both parents reported that they had; they 
said their children were pushed by the program beyond what their school’s band could 
provide, including learning new musical skills and playing more challenging pieces than 
they otherwise would have. One of the parents spoke about how their child was given a 
scholarship to participate in a college band, which they believed would not have been 
possible if not for the Online School Partnerships program. 


Students were asked if they participated in any competitions in the previous school year, 
and 18% reported they had (Figure 21). Of those, 60% had participated in a solo or ensemble 
competition, 29% participated in All-State, and 12% participated in an Honor Band 
competition (Figure 22). 


Students’ musical skills, appreciation of music, and confidence in playing grew as a 
result of participation in the Online School Partnerships program. 


Students participating in the Online School Partnerships program have access to specialized 
and high-level musical instruction that otherwise would not be available to them because 
of cost and the physical distance between their rural communities and instructors who 
specialize in students’ instruments. Teaching Artists and music educators consistently 
described increased access to high-quality music instruction as one of the major successes 
of the program. 

Ina small town, lessons are $15 for a half hour. If there was no grant to help, 


there’s no way. We’re a very poor school — it wouldn’t work. There are maybe 
two families that could do it. —Music educator 


We know that in music education, kids that study privately are going to typically 
be much more successful than those who don't, but it’s a relatively small percentage 
that follow through with private lessons for various reasons, including cost. 
—Music educator 


In interviews, music educators were asked what changes they have seen in students’ 
musical skills and ability due to their participation in the Online School Partnerships. 
Music educators were quite positive about the musical growth their students had seen 
because of their participation in the Online School Partnerships program. Most commonly, 
they highlighted the importance of students learning from someone who has specialized 
in the student’s instrument. They oftentimes spoke about how they are musical generalists 
outside of their own primary instrument, which can be challenging as high school students 
grow on their instruments and could benefit from the technical knowledge of an instrument 
that only comes from specializing in it. 

The horns were really getting into hand placement in the bell... Spending some 

time on that and really thinking about the extent to which their right hand in the 


horn bell impacts their tone quality, their intonation, all that sort of stuff: That’s 
information they maybe wouldn’t have gotten [otherwise]. —Music educator 
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Music educators also spoke about the ways in which the format of the programming leads to 
musical improvements for participating students. Some noted that private lessons are critical 
for students to make substantial improvements in their musical skills, but that very few 
students have that option available to them. Many music educators use the program as a 
way to bring private lessons to these students. Students have additional time with a music 
instructor who is able to listen closely to identify areas for improvement and provide tailored 
feedback; this is not feasible in large group settings like full band or choir practice or even in 
ensembles. By being able to receive tailored feedback, students can progress in their abilities 
and do so more quickly than they can in larger group settings. Music educators also noted 
that Teaching Artists are able to teach students what to listen for in their own playing to 
identify areas for improvement; this skill allows students to make greater progress in their 
own solo practice. 


The vision is that kids who normally wouldn't have access to private instruction 
have access to private instruction. —Music educator 


Additionally, music educators spoke about the ways in which the program has improved 
their students’ confidence in their musical abilities, which in turn improves the quality of 
their music. 


There’s one kid in particular that I’m thinking of. I mean, he’s always played well, 
but it’s just given him a lot more confidence in his ability. — Music educator 


Students show interest and motivation in music. 


Nearly all students reported that they are interested in continuing to develop their musical 
skills (Figure 23). In addition, 70% of students reported that coming to the online sessions 
increased their appreciation of music and 56% reported that they are more motivated to 
do well on their instrument. Thinking of long-term music involvement after high school, 
73% reported they are interested in continuing with their instrument on an informal basis, 
58% in participating in a college level band or orchestra, and 14% in pursuing a college 
degree in music. 


Music educators spoke about the ways in which the Online School Partnerships program 
serves as a motivator for students to do well and continue with their music. One music 
educator spoke about how the program best served students who were eager to improve, 
and specifically to get in to a band that requires an audition, because it gives them the 
tools they need to see marked improvement on their instrument. 


I think it was, in a difficult year, a bright spot. —Music educator 


I have one student that’s taking lessons... she hauls her instrument home every 
day and brings it back, even though I have another one. She’s happy to carry it. 
—Music educator 
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In interviews, parents spoke about how their children’s interest in and dedication to music 
increased because of their participation in the program — by being challenged, students 
were able to see that more was possible for them musically and became more determined 
to do well on their instruments. 


Students gain life skills and show increased interest in school. 


Music educators noted in interviews that they often use the Online School Partnerships 
program as a gifted and talented program for their most committed students; these 
students also tend to be committed to excelling in school. This is supported in student 
survey data, as 95% students agreed they work hard in school (Figure 24). Still, music 
educators noted benefits of program participation that extend to students’ experiences 
within school. They spoke about how the program helps students mature and understand 
life more, as they are exposed to new things and people from a different community that 
is outside of their “bubble.” Music educators also spoke about how students learn to be 
held accountable and to stay on a schedule. They said that these skills and perspectives 
translate to other areas of life, including their performance in and attitude about school. 


Notably, 43% of students reported that participating in Online School Partnerships 
motivates them to do better in school while 25% of students disagreed (Figure 24). This 
level of agreement is not as high as in other questions, which may be attributed to the 
high number of students that agree they work hard in school to begin with and leaves 
little room for improvement due to Online School Partnerships specifically. Similarly, 
about half of students agreed that participating in Online School Partnerships made them 
more excited to go to school (Figure 24). 


Parents were asked in interviews if they had seen changes in their child’s attitude toward 
school and academic skills. Both parents noted that their children were already dedicated 
students, but did speak about how the program motivated their children to push themselves 
to do even better in school and have a positive attitude about tacking academic challenges. 


One noted that the confidence that comes from participation in the program spills over 
into academic performance. 
It’s just that self-confidence... I would read into that, that it would certainly be a 
boost academically. It also instills in them that they have to do hard work to reach 


their goal... It instills in them that if you put in the time and effort, you can succeed 
at whatever it is, whether it’s music or academics. -Parent 


Students are happier and see social benefits from participation in the program. 


In interviews, both parents spoke about how participation in the Online School Partnerships 
program helped their child cope with the isolating aspects of the COVID-19 pandemic. 


This year was very difficult with the pandemic, and she was isolated while 
learning from home. But she looked very happy even though she was practicing 
her instrument for long hours every day. -Parent 
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Issues to consider 


This evaluation of the Online School Partnerships program indicates that the program was 
implemented well and produced positive outcomes. Students generally enjoy their classes 
and are actively engaged and progressing in their music education. Teaching Artists and 
music educators are satisfied with the program and see it as an asset for their schools and 
potentially other schools across Greater Minnesota. Music educators believe that the program 
has increased access to quality, specialized music education for some students who could 
not afford it. 


Based on the results available to date, the following are a few points to consider in 
future programming. 


Continue to provide logistical support to Teaching Artists and music educators; they 
found this to be incredibly helpful and a key reason for the program’s success. 


Find ways for music educators and Teaching Artists to have more time for conversation 
and relationship building, including more in-person opportunities to connect with one 
another once it is safe to do so. Consider ways to build opportunities for students and 
Teaching Artists to meet in-person as well. 


Provide guidance to Teaching Artists about expectation setting with students to ensure 
students are clear about expectations and are being pushed to improve without being 
discouraged or becoming burned out. 


Raise awareness among participants and their families about how to apply for music 
scholarships and how students and their families can set themselves up to be competitive 
for music scholarships. 


Offer financial assistance to families to help them afford musical instruments for 
their child. 
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Results — Sing Play Learn 


The following section reviews the evaluation results of Sing Play Learn with MacPhail® 
Online Early Childhood Music. As mentioned above, the evaluation of this program is 
based on interviews with all three Teaching Artists and surveys with four classroom teachers 
in partnering early childhood classrooms. 


Implementation 


Teaching Artists and classroom teachers identified many benefits of the Sing Play 
Learn program. 


During interviews, Teaching Artists were asked to describe the overall benefits of online 
Sing Play Learn programming. They spoke on the benefits for the partner schools of having 
trained music educators provide classroom music education. While classroom teachers may 
be able to provide some music education on their own, partnering with MacPhail Teaching 
Artists gives teachers access to resources and activities they may not otherwise have access 
to. Since the program is online, it also avoids barriers such as location and transportation. 
Teaching Artists also mentioned that resources or opportunities for music education may 
be limited in smaller towns in greater Minnesota, particularly for this age range. 


Teaching Artists also spoke on the benefits of having music as an additional educational 
component. It can be an added enrichment activity that helps children gain music skills, 
like rhythm and tone, as well as support learning non-musical skills, such as following 
directions and social-emotional skills. This can help support classroom teachers in their 
day-to-day work with students. 


Another mentioned benefit was the impact for the Teaching Artists themselves. One 
Teaching Artist mentioned that Sing Play Learn is an opportunity for them to learn and 
work in a different environment. 


I think, my understanding is that there is not a center like MacPhail in these 
communities, where somebody could be going into the school system to do live 
in-person Classes with the support that I, as a teacher here in the Twin Cities at 
MacPhail have at my disposal... Certainly not to say that there's nobody capable 
in these communities to do musical learning with kids. I think it has to do with 
the support behind what I do, is something that is unique to me being a teacher 
in the Twin Cities, and being able to connect with the kids in out-state Minnesota. 
I continue to be just delighted by how well the online format works, watching 
them build musical skills, which help all the other skills that are on the list of 
what preschoolers are needing to learn, and how they need to develop, is just 
good to see. -Teaching Artist 
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Classroom teachers were similarly asked what they consider to be the greatest benefits of 
the online Sing Play Learn program. All who participated in the survey were positive about 
the impact of the program on the children they teach. They shared the importance of having 
music education so students can learn music fundamentals, and appreciated how the program 
ties into what they’re learning in class. Two teachers also mentioned that the students are 
engaged and look forward to Sing Play Learn. 


Classroom teachers were also asked what they thought were the greatest successes of the 
online Sing Play Learn lessons. Teachers again mentioned that the children are engaged, 
enjoy the program, and look forward to it. One teacher mentioned the importance of young 
children having access to music education. 

The classes are incredibly developmentally appropriate and catered to the needs 


of our classroom. We have really enjoyed being able to connect what we are 
learning in the classroom to different themes in music. —Classroom teacher 


They make it interesting and fun for the students. Classroom teacher 


The Child Development Centers that partner with MacPhail benefit from the 
collaboration. 


Teaching Artists were asked what they believe the benefits of the program are for the 
participating childhood development centers. Teaching Artists most often spoke about being 
an additional support or resource for classroom teachers. For example, classroom teachers in 
preschool often incorporate some musical components on their own. By having set time for 
music with MacPhail Teaching Artists, they are able to dive in further and learn songs or 
activities that are different than the normal day-to-day. Teaching Artists are also able to 
partner with classroom teachers to aid in child development. As mentioned previously, while 
Sing Play Learn may have an impact on music skills, it may also have an impact on non- 
musical skills such as literacy, math, and social-emotional skills. All of these skills are 
important for child development. In this way, Teaching Artists and classroom teachers 
are working together towards a common goal of providing rounded child development 
opportunities. 

A lot of times we ’re working on different skills, school readiness skills, literacy 

skills, impulse control, executive functioning skills. Ideally it’ll fit into what they’re 

already learning at their center... That child development piece is at the center of 


what we do. Of course we want people to know and love music, but we want them 
to be fully developed humans and good citizens of the world too. —Teaching Artist 


In survey responses, all four classroom teachers agreed or strongly agreed that MacPhail 
is a helpful resource for them in their work as an early childhood educator, that they have 
learned new strategies to use music in their classroom, and that MacPhail has provided them 
with new tools and resources to do so. These teachers were asked to describe what strategies, 
tools, or resources they had learned to use in the classroom through Sing Play Learn. 
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Teachers mentioned the value or learning new songs, particularly to help during class 
transitions or grabbing the students’ attention. One teacher mentioned the transition songs 
can be particularly beneficial for students who tend to struggle with transitions. 

We use a few of the songs in the classroom as attention getters or transition songs, 


they are really catchy and the preschoolers enjoy singing along with them. 
—Classroom teacher 


Child Development Centers easily partner with MacPhail. 


All four classroom teachers who participated in a survey agreed or strongly agreed that it 
is easy for them to collaborate with MacPhail; three out of the four strongly agreed. They 
were asked to provide recommendations for how this collaboration could be improved and 
they had few recommendations. One teacher mentioned that it may be appropriate to 
transition to a kindergarten level after Christmas. Another teacher mentioned the switch 
from Skype to Google Meet has been better for their class. 


Teaching Artists and classroom teachers build relationships. 


All Teaching Artists said they have been able to effectively build relationships with 
classroom teachers. They noted that there is often great communication with the teachers 
and that they work together to plan and provide lessons for the children. Two of the Teaching 
Artists mentioned they were originally surprised at how well they’ve been able to build 
relationships in an online format, with one noting they might have an even better relationship 
with their online Sing Play Learn partners than their in-person partners due to the level of 
planning and communication that needs to happen for online classes. 


Teaching Artists offered suggestions for how MacPhail could better support relationship 
building between Teaching Artists and classroom teachers. One suggestion was to have 
more opportunities for in-person connection, not just with the class that’s participating 
but potentially an entire school or community. For example, hosting an event at a local 
community or recreation center and opening it up to everyone. Another suggestion was 

to be more intentional at the beginning or introductory phase of partnership. Beginning a 
new partnership can be difficult or take time, so any additional steps taken to help foster 
that new partnership will be helpful. For example, having a warm hand-off email or kick- 
off meeting between a MacPhail coordinator, the Teaching Artist, and a classroom 
teacher to begin the partnership may help establish better rapport sooner. 
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Teaching Artists and classroom teachers had few suggestions for program 
improvement. 


When asked for suggestions for program improvement, both classroom teachers and 
Teaching Artists had few, if any, suggestions. A few noted that the change from Skype to 
Google Meet has been a good change. A few Teaching Artists noted that they are in the 
process of creating a set curriculum for partner teachers and they are looking forward to 
having that available. 


Outcomes 


Students are engaged in the lessons. 


Classroom teachers were asked if their students are engaged in the online Sing Play Learn 
lessons. Three agreed and one strongly agreed that their students are engaged. Teachers were 
asked in a follow-up question whether or not their students became more engaged in the 
lessons as the year went on. Teachers shared that for the most part, students become more 
engaged as the year went on. They mentioned that as students became more comfortable with 
the activities and expectations, they became more engaged. Three of the four also mentioned 
that their students connected with their Sing Play Learn teacher and looked forward to 
spending time with them. A few did mention that some students struggled to stay on task 
from week-to-week or activity-to-activity. 

Once my students knew what the expectations were during music class, they were 


able to attend better. They love their teacher and look forward to classes with her. 
—Classroom teacher 


Certain weeks were more challenging than others if they would be active on the 
carpet- sometimes towards the end they would be less engaged, but overall they 
are engaged. -Classroom teacher 


Teaching Artists provided a similar assessment of students’ engagement in the lessons. 
They shared that as students become more comfortable with the class and the teacher, they 
become more engaged. Some students that were a bit shy or less familiar at the beginning 
come out of their shell to participate. Students start to ask for certain songs or activities, 
suggesting they feel comfortable and interested enough to take ownership of the class. One 
Teaching Artist shared some students may even wake up from a nap for Sing Play Learn 
because they do not want to miss it. Another Teaching Artist shared that due to COVID-19 
and working from home, the students had an extra opportunity to get to know her and her 
family, which encouraged more engagement and relationship building between her and the 
students. 
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Students show improvements in listening skills. 


Classroom teachers were asked if online Sing Play Learn lessons had improved their 
students’ listening skills and all four agreed that their students’ listening skills had 
improved because of the lessons. Teaching Artists also spoke on changes in listening 
skills, particularly as students become more comfortable and engaged in class. They 
noted that Sing Play Learn develops listening skills because it involves following 
directions, call-and-response activities, and listening closely to a song to follow along. 
These listening skills are important for school readiness and for social-emotional learning. 
There's a very clear line to musical activities promoting those skills. So when we're 
doing an echo song, for example, we are listening to a teacher, and responding, 


they're listening for a melodic or rhythmic pattern, and imitating that directly in 
their response. —Teaching Artist 


Students show improvements in self-regulatory skills. 


Classroom teachers were also asked to what extent they agree or disagree that online Sing 
Play Learn lessons improved their students’ self-regulatory skills. All four agreed with 
the statement. 


Similar to improvements in listening skills, Teaching Artists shared they believe Sing Play 
Learn helps develop self-regulatory skills through the nature of activities. Students are 
building impulse control and executive functioning skills through learning how to follow 
along, echo after someone, and participate in a social activity with others. 


Issues to consider 


This evaluation shows promising results of the program. MacPhail’s Teaching Artists and 
classroom teachers at participating child development centers feel positively about the 
partnerships they have established and the outcomes for participating children. 


Based on these results, the following are actions that MacPhail could consider in future 
programming. 


m= When safe to do so, facilitate opportunities for Teaching Artists, and classes to meet 
in-person to allow for greater relationship building. Consider creating opportunities 
for in-person community events as well. 


= Continue to build structured curriculum to share with classroom teachers as a shared 
resource and avenue to have conversations about shared goals and plans for the 
school year. 
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=m Consider ways to help facilitate relationships between Teaching Artists and classroom 
teachers, with a particular focus on helping these relationships get started at the beginning 
of the school year. 


= Continue to utilize music that is engaging and exciting for participating students. 
Consider incorporating or expanding the use of music from other cultures, including 
the cultural backgrounds of participating students. 


m Consider more ways to encourage family engagement, including virtual events or in- 
person events when it is safe to do so. 


= Consider to have Teaching Artists share musical resources with classroom teachers 
that are age-appropriate and engaging for students. 
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Figures — Online School Partnerships 


Implementation 
3. Students’ general satisfaction with the Online School Partnerships program 
Strongly Strongly Don’t 
N agree Agree Disagree disagree know 
| like coming to the online sessions. 102 20% 63% 5% 2% 11% 
| would recommend these sessions to 96 29% 54% 5% 1% 10% 
other students. 
| am interested in continuing with the 100 13% 30% 24% 2% 31% 


MacPhail Online Partnerships Program. 


Note. Totals may not equal 100 due to rounding. 


4. Students’ satisfaction with technology used 


Strongly Strongly Don’t 
N agree Agree Disagree disagree know 
The technology used for these sessions 96 16% 57% 10% 2% 15% 
made it easy to learn. 
Note. Totals may not equal 100 due to rounding. 
5. Students’ report of instructor 
Strongly Strongly Don’t 
N agree Agree Disagree disagree know 
| liked the Teaching Artist who led the 102 43% 48% 4% 1% 4% 
class. 
The Teaching Artist used class time well. 102 42% 50% 4% 0% 4% 
The Teaching Artist set clear expectations 102 34% 47% 9% 1% 9% 


for practicing and learning at home. 


Note. Totals may not equal 100 due to rounding. 


6. Students’ report of online lessons while distance learning 


Strongly Strongly Don’t 
N agree Agree Disagree disagree know 
It is as easy to do my lessons at home 82 26% 26% 29% 9% 11% 
as it is at school. 
| have space available to do my lessons 82 39% 54% 5% 1% 1% 
at home. 
My family has access to home internet 82 42% 54% 2% 0% 2% 


that is high quality and reliable enough 
for my lessons. 


Note. Totals may not equal 100 due to rounding. 
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7. Students’ satisfaction with Online School Partnerships program classes 


What did you like most about the program? N=63 N Percent 
Improving musical and/or instrumental skills 36 57% 
Positive interactions with the instructor 35 56% 
Lessons were accessible 4 6% 
Other 7 11% 


Notes. Students may have noted more than one thing they liked most about the class, resulting in a total >100%. 
Percentages given reflect the number of times a response was given divided by the total number of responses. 


8. Students’ suggestions for program improvement 


What would you improve about the program? N=54 N Percent 
No suggestions 26 48% 
Changes to lesson or sectional structure 8 15% 
Improving technology or fixing technology issues 5 9% 
Improved organization 5 9% 
Other 11 20% 


Notes. Students may have noted more than one thing they would improve, resulting in a total >100%. Percentages 
given reflect the number of times a response was given divided by the total number of responses. 


Outcomes 


9. Students’ report of musical skills and knowledge 


Strongly Strongly Don’t 
N agree Agree Disagree disagree Know 
| learned new musical skills that 102 28% 56% 8% 2% 6% 


helped me in band/choir. 


Note. Totals may not equal 100 due to rounding. 


10. Tone quality before and after online sessions 


Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post A 12% 32% 46% 10% 0% 3.5 ae 
+0. 
Pre 41 7% 20% 46% 22% 5% 3.0 


Note. Totals may not equal 100 due to rounding. 
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11. Intonation before and after online sessions 


Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 37 14% 30% 49% 8% 0% 3.5 ae 
+0. 
Pre 37 5% 14% 43% 35% 3% 2.8 
Note. Totals may not equal 100 due to rounding. 
12. Rhythm before and after online sessions 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post A 12% 39% 42% 7% 0% 3.6 ae 
+0. 
Pre 4 2% 22% 49% 24% 2% 3.0 
Note. Totals may not equal 100 due to rounding. 
13. Technique before and after online sessions 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post M1 12% 24% 51% 12% 0% 3.4 ay 
+0. 
Pre 4 2% 15% 37% 39% 7% 2.7 
Note. Totals may not equal 100 due to rounding. 
14. Musical expression before and after online sessions 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 41 10% 39% 34% 17% 0% 3.4 9 
+0. 
Pre 41 0% 20% 37% 37% 7% 2.7 
Note. Totals may not equal 100 due to rounding. 
15. Bowing (Strings) 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 5 0% 60% 20% 20% 0% 3.4 6% 
+0. 
Pre 5 0% 20% 60% 0% 20% 2.8 


Note: This skill only applies to students who play stringed instruments, resulting in a smaller N. Totals may not equal 
100 due to rounding. 


MacPhail Online School Partnerships 2020-21 Evaluation Results 27 | Wilder Research, December 2021 


16. Execution 


Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 40 13% 35% 45% 5% 3% 3.5 64 
+0. 
Pre 40 0% 20% 43% 33% 5% 2.8 
Note. Totals may not equal 100 due to rounding. 
17. Articulation 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 39 5% 39% 46% 10% 0% 3.4 ais 
+0. 
Pre 39 5% 18% 39% 36% 3% 2.9 
Note. Totals may not equal 100 due to rounding. 
18. Reading skills 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 39 8% 28% 39% 26% 0% 3.2 re 
+0. 
Pre 39 5% 23% 28% 39% 5% 2.8 
Note. Totals may not equal 100 due to rounding. 
19. Confidence 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post A 17% 41% 39% 0% 2% 3.7 ae 
+0. 
Pre A 5% 27% 37% 29% 2% 3.0 
Note. Totals may not equal 100 due to rounding. 
20. Ability to focus 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post A 29% 46% 20% 5% 0% 4.0 a 
+0. 
Pre A 17% 46% 24% 10% 2% 3.7 


Note. Totals may not equal 100 due to rounding. 
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21. Students’ report of participation in musical competitions 


N Participated in a competition 
93 18% 


22. Students’ report of types of musical competitions participated in 


N Solo/Ensemble Honor Band All-State 
17 60% 12% 29% 


Notes. Students could check all that apply, resulting in a total >100%. Percentages given reflect the number of times a 
response was given divided by the total number of responses. 


23. Students’ report of musical interest 


Strongly Strongly Don’t 

N agree Agree Disagree disagree know 
Coming to the online sessions increased 102 21% 49% 11% 1% 19% 
my appreciation of music. 
| am interested in continuing to develop 99 43% 51% 0% 0% 6% 
my musical skills. 
| am interested in pursuing a college 100 4% 10% 39% 18% 29% 
degree in music after graduating from 
high school. 
| am interested in participating inacollege 100 21% 37% 10% 5% 27% 
band, orchestra, or choir after graduating 
from high school. 
| am interested in continuing with my 100 22% 51% 5% 3% 19% 
instrument on an informal basis after 
graduating from high school. 
| am more motivated to do well on my 96 16% 40% 23% 3% 19% 
instrument. 
Note. Totals may not equal 100 due to rounding. 

24. Students’ report of work in school 
Strongly Strongly Don’t 

N agree Agree Disagree disagree know 
| work hard in school. 96 56% 39% 0% 0% 5% 
Participating in the Online School 97 10% 33% 25% 9% 23% 
Partnerships made me more motivated 
to do well in school. 
Participating in the Online School Program 96 9% 43% 21% 5% 22% 


made me more excited to come to school. 


Note. Totals may not equal 100 due to rounding. 
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Appendix 


Open-ended comments 


A1. Student survey open-ended comments: “What did you like most about the program?” 


All Schools (N=63) 


Actually playing along with the person. 

Being able to join from home. 

Don't know 

Getting better at my instrument 

He gave good insight on practice and practice skills. 


How | learned how to fix my errors in order to correct myself and learn ways to get better from making 
those errors. 


How | was able to get things that | wanted to get done and | had control over what was being done. 
How it got everyone involved 

How we got to have someone specialized in our instrument. 

| enjoyed getting some insight into my instrument that | didn't know before. 

| enjoyed improving difficult music the most. 


| enjoyed the interaction between the instructor and me, which allowed me to learn more quickly than in 
a full band. 


| enjoyed the one-on-one time within the sectional. 

i learned a lot 

| learned how to play my instrument better. 

| learned more about my instrument and how to hold/play it better 
| like how accessible music mentorship is. 


| like how the teacher's lessons depended on what questions were on the minds of their students in the 
sessions. They fit their agenda to answer any unsolved questions the students had. 


| like our teacher because he was upbeat and fun. 

| like that | didn't have to go way out of my way to get lessons. 

| like the amount of work | am given each week and the way my teacher is instructing me. 

| liked being able to hear how the teacher was playing while he was also explaining it. 

| liked being able to learn new things. 

| liked hearing about the experience of our master class teacher and the advice that she had to offer. 
| liked hearing from a professional musician. | am more proficient now then before the classes. 

| liked how clear the instructions were and how the teacher used class time efficiently 

| liked how even though we were distanced you still grow a connection with your lesson teacher. 


Note. Responses may have been slightly edited for clarity and confidentiality. 
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A1. Student survey open-ended comments: “What did you like most about the program?” 
(continued) 


All Schools (N=63) 


| liked learning about little tips and tricks to help me play faster and clearer. 
| liked my teacher and how he taught. 


| liked that the teacher helped answer all my questions which help improved me playing and that | also 
learned new things about my [instrument]. 


| liked the instructor, he was nice and had some good information. 
| liked the [teaching artist]. Also | like some of the warmup things that he taught us 


| loved learning new skills and techniques with my instrumental as well as new things about music theory. 
My teacher was also very nice and motivational. 


| mostly liked the one on one time that helped me improve my skills as an individual. 

| only went to one session, but it was fun and i enjoyed it 

| think it is a great program for people who are more committed to music than | am. 

It gave a good sense of individual help with different parts of overall playing and technique. 
It helped me gain confidence in my abilities. 

It helped me get better at the music | was playing and was easy to do. 

It helped us improve on what we needed to work on. 


It taught me skills that | never knew I'd need. They also helped teach me how to tongue on my instrument 
when | never knew how. They also would give me tips on music that wasn't my solo to help me in concert 
band as well. 


It was a very solid base for me to use. 
It was informational and gave me good advice. 
just how quickly the instructor picked up on our skill level and based the sectional on our level of skill 


learning more skills and techniques on the vibraphone that | can use to learn more solos for fun or performing 
and that | can use college to be a good asset to the band. 


My [teaching artist]! She’s THE BEST! SO SWEET! so encouraging and helpful!! 

[Teachign artist] is a very good teacher. 

Not sure. 

Sectionals with the class were a lot of fun. In a way, it felt very personable even with the online barriers. 
That it was with just the Trumpet section so we could work better. 

The ability to have a near one on one connection with an expert was great. 

The instructor was nice 

The instructor was very engaged and offered technological tools to improve my musicianship. 

The skill that we’re taught 

The teacher taught us about playing in tune by showing us how to adjust the mouthpiece of our instruments. 
The teachers and being also to learn stuff | wouldn’t in school 


Note. Responses may have been slightly edited for clarity and confidentiality. 
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A1. Student survey open-ended comments: “What did you like most about the program?” 
(continued) 


All Schools (N=63) 


The teachings 
The thing that | liked in the program is the overall little tips and tricks about my sections instrument. 


This program helped me learn and improve, and | still had fun and was able to understand how to do 
new things. 


We played along with the teacher in exercises and pieces. We did a lot of different things to improve our 
playing. 

We were taught better techniques for articulation 

What they teach us and ways we can get better with our instruments 

Working with [teaching artist] was super fun and | learned a ton. 


Note. Responses may have been slightly edited for clarity and confidentiality. 
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A2. Student survey open-ended comments: “What would you improve about this 
class?” 


All Schools (N=54) 


Being a little more patient while doing the lesson 

Better educate the teachers about the technology used in online school. 
Can not think of anything 

Don't know 

Either a bit more time or more efficient use of time. 

Get everyone EVERYONE in the group involved and feel welcomed 
Have a better planned out agenda for the session. 

How disorganized it was 

| can't think of anything at the moment. 

| can't think of anything right now because | joined a little later and not for very long. 
| do not have any input about the improvement of the program. 

| don't have any ideas for improvement. 

| don't have any improvements to make. 

| don't know 

| don't know 


| mean learning from home was less than ideal but after Covid is all over it will be cool to learn from the 
dudes in person. 


| really liked the program and | can't think of any way on making it better. 

| think this program is really good and doesn't need much more improving. 

| thought it was all very good. 

| would change how confusing the dates are. A better calendar or something to see when | have lessons. 
| would devote more time to lessons, and less times to him just playing things 


| would do more centered around how to do specific things from the song we are playing instead of just 
general things. 


| would have like to have a more focused rehearsal either on marching or concert percussion. 
idk 
I'm not sure. 


It might be more rewarding to have more people in our section when we do the master class or meet 
people from our section at other schools. 


it’s a little pricey 
Make sectionals a little longer so there's more time for individual growth. 
Make the time longer 


Note. Responses may have been slightly edited for clarity and confidentiality. 
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A2. Student survey open-ended comments: “What would you improve about this 
class?” (continued) 


All Schools (N=54) 


Maybe the instructors knowledge on the programs they are using like google meet, at the beginning it 
was a bit technological rough, but it all worked out, but other than that | have nothing else 


Maybe what is taught in the time frame could be more useful. 

Maybe work on a wider variety when working with sections 

More consistency and better communication. 

More interactive, we just kind of sat there, and did what he said, but it wasn't that bad 
More specific technique advice for individual students 

N/A 

N/A 

N/A 

Not sure. 

Not sure. 

Not sure. 

Nothing 

Nothing 

Nothing because they did the best they could with the circumstances. 
Nothing really 

Nothing really. 

Nothing, | believe it’s a great program 

Nothing. It worked well when the internet was working. 

One on one time would probably be more impactful than sectionals. 
One thing that this program can do better is a little more playing during the session. 
Probably a little more time for the lessons. 


The only thing | didn’t enjoy was that it was online because there was some lag and audio issues. We 
would sometimes freeze on both sides which made it a little frustrating. 


The teacher had internet issues and was not present for most of the meetings. 
Unfortunately, nothing can beat an in-person lesson. 


Note. Responses may have been slightly edited for clarity and confidentiality. 
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Logic model 


MacPhail Online School Partnerships Logic Model 


Overview: The MacPhail Online School Partnerships was established to partner with schools in greater Minnesota to support their music program’s 
goals by providing MacPhail faculty to teach students remotely. By teaching classes remotely, MacPhail aims to increase students’ skills, knowledge, 


and interest in music, and provide the resources for music educators to grow professionally. 


INPUTS > ACTIVITIES > OUTCOMES 


Teaching artists 


Videoconferencing 
technology 


MacPhail funding 


Other funding 


Facilities and 
physical space 


Students 


Music educators 


Teaching artists work with music educators 


to determine curriculum for students 


SHORT TERM 


LONG TERM 


Tailored lessons are offered 
to students in addition to their regular 
music classes, which include: 
- Group lessons 
- Sectionals 
- Individual lessons 


Student outcomes 


- Students show improved musical skill and 
knowledge'34 


- Students show increased interest, appreciation, 
and motivation in music? 


- Students show improved attitudes toward school? 
- Students show growth in non-academic skills? 


Students continue their 
music education 
throughout high school 
and in college® 


Student musical performances 


Music educator outcomes 
- Music educators grow as teachers* 


- Music educators have increased resources and 
capacity to grow their music programs* 


Rural students are more 
likely to have the 
opportunity to perform 
and compete at a high 
level? 


Music educators observe Teaching Artists 
and learn of new teaching approaches, 
methods, and resources 


MacPhail outcomes 


- MacPhail is able to reach out to students who would 
otherwise have no access to the organization® 


- MacPhail creates partnerships with schools 
throughout the state*> 


Partner schools 
throughout the state are 
better able to offer high- 
quality music education 
to their students® 


Evaluation and documentation 


MacPhail Online School Partnerships 2019-20 Evaluation Results 


1 Measured by music skills assessment 
2 Measured by progress reports 
3 Measured by student survey 


4Measured by music educator survey 
5 Measured by Teaching Artist survey 
8 Measured by program records 


35 | Wilder Research, October 2020 


Survey instruments 


MacPhail 


CENTER FOR MUSIC 
Student(s): 

School: 

Director: 

Instrument{s): 


Please place one of these numbers in each box below; then total. 5 = superior; 4 = excellent; 3= good; 2=fair; 1=poor 


Performance Factors 


MacPhail Teaching Artist: 


Comments 


Tone Quality 
Clarity, consistency, control, 
focus, warmth 


Intonation 
Accuracy, consistency of pitch 


Rhythm 

Accuracy of note and rest 
values, correctness of meters, 
duration, pulse, steadiness 


Balance Blend 
Accompaniment, awareness 
of ensemble, likeness of 
qualities, matching of 
registers (if applicable) 


Technique (facility/accuracy) 
Attacks, releases, control of 
ranges, mechanical skill 


Interpretation/Musicianship 
Dynamics, emotional 
involvement, artistry, 
ohrasing, style, tempo 


Diction — Vocal 
Bowing — Strings 
Articulation — Winds 
Execution - Percussion 


Other Performance Factors 
General conduct, 
mannerisms, poise, posture, 
etc. 


TOTAL POINTS 


Signature of Teaching Artist: 


Date: __ 
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MacPhail Online School Partnership Student Survey 


Spring 2021 


Name: 


School: 


Grade: 


Choir/Band Teacher: 


Where do you do your online sessions? 
(Home, school, both): 


Type of session (individual, sectional, or 
class): 


This survey will ask you questions about you and the music sessions you participating in this semester. 
We are interested in your honest thoughts and ideas, so there are no right or wrong answers. 


Your answers are important. Please indicate how much you agree with each of these statements and 


fill out all parts of the survey. 


Strongly Strongly Don’t 
disagree Disagree Agree agree Know 

1. |learned new musical skills that helped Ch mE mE [4 [Js 
me in band/choir. 
| like coming to the online sessions. LI Ll Ls Ls Ls 

3. | liked the Teaching Artist who led the C1 Ele [3 []4 [Js 
class. 

4. Coming to the online sessions increased Ch [2 mE []4 [Js 
my appreciation of music. 

5. The Teaching Artist used class time well. LI Ll [ls Lil B. 

6. The Teaching Artist set clear expectations C mE mE [|4 [s 
for practicing and learning at home. 

7. | am interested in continuing with the Ci mE mE [4 [Js 
MacPhail Online Partnership program. 

8. | am interested in continuing to develop CI) [2 C3 []4 [ls 
my music skills. 

9. | am interested in pursuing a college 
degree in music after graduating from high LI LI ER Ls Ls 
school. 

10. | am interested in participating in a college Oh ye ee at ze 


band, orchestra, or choir after graduating 
from high school. 


MacPhail Online School Partnership Student Survey 


Wilder Research, January 2021 


Strongly Strongly Don’t 
disagree Disagree Agree agree Know 


11. | am interested in continuing with my 
instrument on an informal basis after LI Ee im, Lis L|p 
graduating from high school. 


12. Because of Online School Partnerships, | 


am more motivated to do well on my LI El i, Ei Ls 
instrument. 

13. The technology used for these sessions C1 [2 Be []4 [ls 
made it easy to learn. 

14. | would recommend these sessions to CI [lz mE []4 [ls 
other students. 

15. | work hard in school. LI} LI} Ls ils Ls 

16. Participating in the Online School 
Program made me more excited to come Ly} Bi LP Eile Ls 
to school. 


17. Participating in the Online School 
Program made me more motivated to do bls [sl LJs L|4 Ls 
well in school. 


[For students who indicate they take lessons at home] 
Please indicate how much you agree with these statements. 


Strongly Strongly Don’t 
disagree Disagree Agree agree Know 
18. It is as easy to do my lessons at home as i 
it is at school. Ly Ll} Ls Lh a 
19. | have space available to do my lessons at ? is ’ ‘ 
sets LJ LJ LI LJ LJ 
20. My family has access to home internet 
that is high quality and reliable enough for LI LJ LJ: [ls Lis 


my lessons. 


21. What did you like most about the program? 


22. What would you improve about the program? 
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23. Did you participate in a competition this year? 
L1' No 
L|? Yes 
23b. [If yes]:Which competitions did you participate in? (Check all that apply) 


LJ? — Solo/Ensemble 

L14 Honor Band 

LIr All-State 

Ls Other (please specify): 


THANK YOU FOR COMPLETING THE SURVEY! 
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